Field of the Invention
The method and apparatus described herein are motivated by the need for surveillance and security of the land borders of the United States, including property and facility borders such as military bases and nuclear/petroleum facilities, respectively. The border surveillance system will detect, track, and/or classify people or vehicles automatically and in real-time that are approaching, that may attempt to cross, or have crossed a border. The surveillance method is comprised of (1) a radar for wide-area surveillance to detect, track, and perform 1st stage classification of potential targets that cues (2) one or more optical sensors, which may be comprised of one or more infrared (IR) and electro-optical (EO) sensor systems, and/or coherent radars, for classification of these targets, and (3) a supercomputer such as an IBM® Cell Broadband Engine® supercomputer to process the collected data in real-time. The preferred embodiment is a non-coherent radar cueing a long wavelength IR (LWIR) sensor system, where the non-coherent radar can also be operated as a stand-alone system.
The detection, tracking, and classification will be facilitated by advanced signal processing techniques that allow the use of less expensive and off-the-shelf radars and sensor systems, and at the same time achieve better performance, increased range, and automatic decision making than these off-the-shelf systems. The sensor systems can be implemented on one or more tower-like structures spaced appropriately to cover the border sector of interest, or a mobile vehicle designed to move the surveillance system to a particular location of interest, where it can be used alone or in combination with the tower system. Detection, tracking, and classification are done automatically in real time, and suspected targets are confirmed by visual inspection of the sensor output images by an operator before resources are committed for apprehension. A low-cost tracker UAV can be launched to track the designated target until apprehension. The method includes the sensor suite, the operational strategy, the signal processing approach, the signal processing hardware, and the signal process algorithms. The method has homeland, military, energy, property, building, and facility security applications.
Brief Description of Prior Art
The method and apparatus of the present invention is motivated by the need to secure the borders of the United States of America from illegal entry in a cost effective, reliable manner. The invention proposed herein is further motivated by the need to rapidly, accurately, automatically, and cost effectively detect, track, and classify people and vehicles in real time with a fully integrated system. The invention proposed herein is a smart sensor system that can be used to address this surveillance problem using one or more of the system implementation architectures described herein. The same system or subsystems (e.g., the non-coherent radar acting as a stand-alone system) can be used to address a wide range of surveillance and security applications, including military, industrial, and commercial one. For example, a smart sensor system can be used in addition to country borders, for example, to determine all people and vehicles (including aircraft) approach a secure military base, a nuclear power plant, a crucial commercial asset (e.g., water supply), a U. S. Department of Energy (DOE) facility, or a petroleum facility/pipeline.
The problem and the significance of the border surveillance problem for the United States are clearly illustrated in a report issued by the Department of Homeland Security (DHS) in December 2005 [1]. As an example, the Southern Border between Mexico and the United States is 2,000 miles long. The DHS estimates that there may be 3,000 illegal aliens per day crossing the Southern Border, and the report indicates that the border patrol agents are responding to an unwieldy number of possible crossings, one every 44 s [1]. The existing sensor system used at the borders is comprised of seismic and magnetic sensors, and the remote video surveillance (RVS) cameras, which are not cued to the sensors. The report suggests that the probability of false alarm of the system is very high (between 34% and 96% of the sensor alerts) and the probability of detection is very low (between 1% and 57%) [source: Table 1, page 22 of [1]]. Of the seven recommendations made to improve border surveillance, four were technologically driven. They included better integration of assets, standardization of the data collection, processing and reporting, development and application of performance measures, and identify and deploy the use of non-permanent or mobile surveillance platforms. The method and apparatus of the present invention addresses all of these recommendations.
The inadequacy of the current surveillance sensor algorithm and automatic false alarm mitigation technology, the excessively large number of illegal aliens attempting to cross the border, and the lack of sufficient agents to verify possible crossings and to apprehend the illegal aliens makes this a very difficult problem. The invention proposed herein provides the needed surveillance tracking and false alarm mitigation technology that is real-time, integrated, automatic, and has the range to allow the border patrol agents to avoid wasting time on false alarms, enables operators to quickly validate real-targets and determine in advance if they are safety threats, and projects real target locations to future apprehension points. This can be accomplished by the right combination of commercially available off-the-shelf (COTS) sensors and advanced signal processing algorithms.
The report further states that the existing RVS cameras do not automatically steer and cover the area where a seismic or magnetic sensor alarm occurs and do not have the ability to detect movement automatically without operator input. Thus, with existing systems being used at the U.S. borders, it is possible for illegal activity to go unnoticed if an agent is not manning the video terminals or misses some activity on the screen. This can be addressed by used of signal processing algorithms that automatically detect, track, and discriminate human activity without the need for operator involvement on the screen.
The report also states that with existing systems, weather conditions and power outages can have a significant impact on the performance of the systems. While the ice and snow can impair visual acuity along the Northern Borders, the hot and humid conditions can impact the performance of the RVS cameras and the resolution of the IR systems. Other sensors, like an X-band tracking Radar that, unlike higher frequency systems that are impacted by weather, has demonstrated reliable performance in high humidity, rain, and ice/snow conditions. In addition, overlapping Radar coverage is needed such that coverage integrity is maintained with the loss of adjacent sensors, and is enhanced with diverse Radar look directions in difficult terrain conditions.
The method and apparatus of the present invention address some of the most significant deficiencies in the current border operations. The poor performance of the present technology-based systems is due partially to the lack of integration between the sensors and the RVS cameras, the high number of false alarms of the sensors, and the lack of automation for the present camera systems. The 24/7 requirement for visual inspection of the camera images by an operator, and the requirement to deal with a sensor alert every 44 s is too people-intensive to work over an extended period of time. Automation can reduce the workload and take better advantage of an operator's inherent ability to discriminate threats from false alarms. It is clear that better coverage, better integration (cueing of sensor and RVS camera resources), the use of advanced signal processing algorithms for automatic detection, and improved operator discrimination can improve the performance of the system.
The method and apparatus of the present invention will increase the PD, reduce the PFA, reduce the PFT (i.e., number of false agent prosecutions after final operator screening), reduce the total number of alerts to deal with, and allow operators to be more effective. The proposed method enables more effective sensor assets because of better integration, automation, and better coverage. The method and apparatus of the proposed system architecture can be integrated with the existing underground seismic and magnetic sensors, the RVS cameras, and other assets presently being used by DHS.
Detection of small cross-section targets such as people or groups of people that can potentially cross a border, especially one that is large, is a challenging technical problem if it is to be accomplished in a cost effective method. The problem is made even more difficult when the measurements need to be made at ranges up 10 km or more for detection, tracking, and classification. People, with very cross sections (less than 1 m2) are very difficult to detect. The Department of Homeland Security has been using unattended ground sensors, EO/IR cameras mounted on towers and in UAVs for visual identification and classification of people and vehicles (at short ranges) by an operator. However, as described below, this approach has experienced problems with an unacceptably high false alarm rate and a poorer than desired probability of detection. The former results in an expensive misuse of limited resources, and the later does not provide the desired border security.
The invention proposed herein whereby a radar is used for wide area surveillance to detect and track potential targets in an area of interest and then cues an optical system such as a long wavelength IR sensor for classification has never been used for border surveillance. The use of the radar allows targets to be detected at sufficient ranges so that potential border crossings can be anticipated in sufficient time for an optical system to identify the target before an illegal crossing is attempted. The invention proposed herein also describes a two-zone implementation, one for wide area surveillance and one for classification and continued tracking until the target is dismissed as a potential problem or through apprehension. The towers or vehicles presently used by DHS do not include radars and consequently have not used such radars to cue the EO/IR sensors. For this border surveillance architecture to be effect requires that the radar have a wide area surveillance capability with frequent updates of the potential targets moving towards the border; this requires many millions to tens of millions of radar samples be collected over the target area. For complete surveillance, the radar should be preferably implemented with 360-degree, 24-h-per-day coverage.
Most modern-day radars developed by the military are coherent radars, which detect moving objects such as people, vehicles, aircraft, ships, and missiles using range-Doppler processing. However, these radars are very expensive and to realize acceptable coverage for border surveillance applications would generally not be cost effective. The less expensive non-coherent radars have not been used for border surveillance applications because of the lack of capability to detect very small cross-section targets like people. Incoherent radars have mainly been used to detect threats much larger than people like aircraft and ships. FIG. 1 illustrates this point with a Furuno® radar mounted on a ship (insert) that has been used for marine applications for many tens of years. These low-cost, non-coherent radars operating at X-band, are used to detect buoys and land and have not been used for detection and tracking of people. FIG. 1 shows a typical output display of such a radar in a port region where people, vehicles, ships, buoys, and land are present, but only the land and buoys are visible. In fact, no people are visible in the display and no attempt has been made to use this type of radar for people detection. (For comparison purposes, contrast the output display from FIG. 1 to the raw data shown below in FIG. 17.)
Besides low cost, these radars have sufficient range and an effective wide-area surveillance capability to cover and provide a large number of samples on the target area of interest. The advantages of such a non-coherent radar is that it is very inexpensive, can put a lot of energy on the target, and can collect a lot of samples on the target. This type of non-coherent radar is the key element of the preferred embodiment of the present invention proposed herein because the advantages of this type of radar, when applying advanced signal processing, can be converted into a radar to detect targets with very small cross sections (i.e., people). The invention proposed herein also describes the use of a radar propagation model, which includes the effects of multi-path, to determine signal-to-noise ratio (SNR) of for each resolution cell in the radar target area of interest, where the terrain (in three dimensions), the radar location and elevation, and the target size and height is input to the model. This model can be used (1) in optimally locating the position and elevation of the sensor measurement systems (towers and mobile assets) for maximum coverage and performance, and (2) in developing and maintain a target track even when the target passes through resolutions cells without sufficient SNR for detection.
Our search of the patent literature does not indicate the use of non-coherent for wide area surveillance of people and other targets with small cross sections. Nor does our search indicate the disclosure of the system architecture whereby a radar cues an optical system, the use two-zone method for wide area surveillance in the outer zone and classification in an inner zone. Finally, our search did not indicate the use of a radar propagation model for locating towers or mobile vehicle for optimal coverage, or for use in developing more robust tracks of the targets of interest.
While the use of advanced signal process algorithms and supercomputers automated, real-time measurements with remote sensor systems is not in and of itself novel, it is novel for application to inexpensive non-coherent radar system for detection and tracking people or groups of people.